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We  used  the  statistical  technique  of  factor  analysis  to  design  binary  and  ternary 
optical  correlation  filters  to  identify  objects  in  the  presence  of  unknown  or  non- 
repeatable  distortions.  We  considered  values  of  spatial  frequencies  of  training  set 
imagery  as  features  and  used  those  spatial  frequencies  in  our  filter  depending  on 
their  variation  across  a  training  set.  In  addition,  we  provided  general  expressions 
for  performance  measures  as  a  function  of  training  set  imagery.  The  potential  of 
our  approach  was  evaluated  with  infrared  sensor  imagery  that  varied  in  an  unknown 
way.  Our  statistically  designed  filters  were  easily  calculated  and  performed  well 
in  the  presence  of  noise.  Furthermore,  the  performance  of  our  filters  were  varied 
by  allowing  for  trade-offs  in  performance  measures.  Our  filters  reduced  the 
sensitivity  of  binary  and  ternary  phase-only  filters  to  changes  in  an  object's 
appearance  when  the  input  imagery  varied  in  an  unknown  manner. 
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Abstract 


We  used  the  statistical  technique  of  factor  analysis  to  design  binary  and  ternary  t^tical 
conrelation  filters  to  identify  objects  in  the  presence  of  unknown  or  nonrepeatable  distor¬ 
tions.  We  considered  values  of  spatial  frequencies  of  training  set  imagery  as  features  and 
used  those  spatial  frequencies  in  our  filter  depending  on  their  variation  across  a  training 
set  In  addition,  we  provided  general  expressions  for  performance  measures  as  a  function 
of  training  set  imagery.  The  potential  of  our  approach  was  evaluated  with  infirared  sensor 
imagery  that  varied  in  a  unknown  way.  Our  statistically  designed  filters  were  easily  calcu¬ 
lated  and  performed  well  in  the  presence  of  noise.  FurtheniK>re,  the  performance  of  our  fil¬ 
ters  were  varied  by  allowing  for  trade-offs  in  performance  measures.  Our  filters  reduced 
the  sensitivity  of  binary  and  ternary  phase-only  filters  to  changes  in  an  object's  appearance 
when  the  input  imagery  varied  in  an  unknown  maimer. 
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1.0  Introduction 


The  primary  difficulty  with  the  use  of  binary  phase-only  filters  (BPOFs)  with  sensor 
imagery  in  an  optical  correlator  is  the  filter’s  sensitivity  to  changes  in  a  object's  appear¬ 
ance.  In  addition  to  these  geometric  distortions,  the  image  of  an  object  at  a  specific  orien¬ 
tation  can  change  due  to  effects  such  as  lighting,  weather  and,  especially  in  infirared 
imagery,  the  temperature  differences  between  components  of  the  object  The  result  is  that 
object  boundaries  are  often  ambiguous  or  poorly  defined  in  sensor  imagery.  If  the  global 
shape  of  an  object  is  used  for  recognition,  then  changes  in  the  imagery  are  often  enough  to 
significantly  degrade  the  performance  of  a  binary  optical  correlator,* 

Because  imagery  firom  sensors  often  changes  in  an  unknown  way,  a  large  number  of 
reference  images  are  needed  to  match  an  image  in  many  applications  when  using  a  BPOF. 
Therefore,  in  an  effort  to  make  a  binary  optical  correlator  more  practical,  optical  correla¬ 
tion  filters  that  recognize  objects  varying  in  an  unknown  way  is  needed;  filters  should  rec¬ 
ognize  objects  in  a  training  set  as  well  as  outside  the  training  set 

Much  work  has  been  performed  in  designing  distortion-invariant  correlation  filters,^"^ 
but  fewer  methods  exist  for  creating  distortion-invariant  BPOFs.^"^  Generally,  these 
approaches  have  been  primarily  used  to  identify  or  reject  geometrically  transformed  ver¬ 
sions  of  an  object  or  to  be  robust  in  the  presence  of  noise.  Using  these  techniques,  filters 
can  be  created  that  can  perform  well  when  the  distortion  can  be  well  characterized. 

Statistical  approaches  to  distortion-invariant  filter  design  have  been  previously  used. 
For  example,  Kumar  et  al.  modeled  image  differences  of  a  training  set  as  a  stochastic  pro- 

o 

cess  to  design  an  optimal  filter.  In  another  statistical  approach,  a  linear  transformation 
matrix  was  used  to  cluster  all  training  vectors  from  each  class  to  a  single  location  in  a 
reduced-dimensionality  signal  space.^  The  variance  of  time-sequenced  correlation 
responses  also  has  been  examined  to  create  a  distortion-invariant  filter.***  However,  by 
examining  only  the  maxima  of  cross-correlation  responses,  little  can  be  said  about  the 
behavior  occurring  at  individual  spatial  frequencies. 

Correlation  filters  can  achieve  improvements  in  signal-to-noise  ratios  (SNRs)  by  set¬ 
ting  the  filter  to  zero  at  particular  spatial  frequencies,  which  creates  a  ternary-valued  fil- 
ter.  ’  ^  In  addition,  this  approach  has  had  success  in  improving  the  distortion  range  and 
SNR  of  distortion-invariant  BPOFs.*^’*^  These  methods  set  filter  pixels  to  zero,  based  on 
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the  ratio  of  the  spectra  of  the  noise  to  the  composite  filter  image  exceeding  a  specified 
threshold. 

In  contrast  to  previous  attempts  at  distortion-invariant  binary  and  ternary  phase-only 
filter  (POF)  formulation,  we  developed  a  filter  using  a  statistical  technique  whose  values 
were  determined  by  features  of  the  training  set  We  attempted  to  find  a  filter  that  repre¬ 
sented  the  critical  characteristics  of  an  object  so  that  objects  outside  the  training  set  could 
be  identified.  We  examined  an  ensemble  of  BPOFs  to  select  spatial  frequencies  that  would 
recognize  images  outside  of  our  ensemble.  Because  correlation  filters  are  derived  from  the 
Fourier  transform  of  an  object,  we  examined  feature  extraction  in  the  Fourier  domain.  We 
considered  the  BPOFs  of  input  images  as  a  set  of  features  to  recognize  objects  and  used  a 
statistical  approach  to  examine  features  that  were  invariant  with  respect  to  the  training  set 
Similar  approaches  have  been  used  to  perform  feature  extraction  with  optical  proces¬ 
sors.  We  retained  those  Fourier  features  that  were  invariant  among  a  training  set  and 
set  to  zero  those  that  varied  using  a  technique  similar  to  factor  analysis  to  design  a  ternary 
filter. 

In  the  next  section  we  describe  our  approach  for  the  statistical  design  of  distortion- 
invariant  binary  and  ternary  POFs  based  on  features  of  a  training  set  Section  3  contains 
computer  simulations  using  our  statistically  designed  filter,  along  with  results  from  our 
attempt  to  increase  the  SNR.  Finally,  our  conclusions  follow  in  section  4. 

2.0  Statistical  binary  and  ternary  phase-only  filters 

2.1  Filter  development 

We  used  features  of  training  images  to  design  both  binary  and  ternary  POFs.  To  con¬ 
struct  our  filter,  we  considered  a  signal  space  where  each  feature  of  an  image  was  repre¬ 
sented  as  an  axis.  The  position  along  an  axis  represented  the  value  of  that  feature.  For  a  set 
of  training  images  in  the  same  class,  we  examined  the  variance  of  the  values  of  each  fea¬ 
ture.  We  eliminated  features  associated  with  large  variances  by  setting  the  value  of  that 
feature  to  zero.  In  this  way,  we  formed  a  cluster  of  training  images  in  a  reduced  dimen¬ 
sionality  feature  space.  We  then  chose  a  set  of  features  that  best  represented  our  cluster. 

Because  BPOFs  of  objects  have  been  sufficiently  descriptive  in  many  binary  optical 
pattern  recognition  experiments,  we  considered  the  BPOF  of  a  training  image  as  a  coUec- 


4 


don  of  characterisdc  features.  Therefore,  each  axis  in  our  signal  space  was  represented  by 
a  discrete  spatial  frequency.  The  position  along  an  axis  represented  the  coefficient  of  the 
discrete  Fourier  transform  (DFT)  at  that  spatial  frequency. 

There  are  different  ways  to  create  a  BPOF  of  an  object.**’^^  We  used  a  comiiu>n 
method  to  create  BPOFs  where  the  phase  was  set  to  +  1  at  each  pixel  according  to 


liyO  <  (j)^  [rt,  m]  <  jc 
-Ir/jt  ^(j).  [n,  m]  <2ji 


(1) 


where  H^n/n]  was  the  BPOF  of  the  ith  training  image,  was  the  phase  of  the  com¬ 
plex  conjugate  of  S,[/i,m],  and  S,[rt,?n]  was  the  DFT  of  the  ith  training  image.  If  the  calcu¬ 
lated  phase  angle  was  between  0  and  n,  the  phase  at  that  pixel  was  set  to  -i-l.  If  the 
calculated  phase  angle  was  between  ic  and  27C,  the  phase  at  that  pixel  was  set  to  -1. 

The  Fourier  coefficients  of  the  training  images  were  examined  over  the  training  set  on 
a  pixel-by-pixel  basis  in  terms  of  their  similarity,  with  the  distance  in  signal  space  between 
Fourier  coefficients  as  a  measure  of  their  similarity.  The  smaller  the  distance  the  more 
similar  the  coefficients.  Because  we  used  BPOFs,  only  two  possible  values  existed  for  an 
individual  pixel.  If  values  of  two  BPOFs  corresponding  to  the  same  pixel  were  the  same, 
we  described  the  distance  between  them  as  -1;  if  the  values  were  different,  we  considered 
the  distance  as  +1.  The  distances  between  coefficients  of  two  BPOFs  were  described  as 


d-  j  [n,  m]  =  H.  [n,  m]  ®Hj  [n,  m] , 


(2) 


where  H^n,  m]  and  Hjin,  m]  represented  two  BPOFs  of  the  training  set,  n  and  m  were  dis¬ 
crete  spatial  frequencies,  and  ©  was  the  exclusive-OR  (XOR)  function.  The  coefficient  at 
each  pixel  of  the  BPOFs  are  +1;  therefore,  we  used  +1  in  Eq.  (2)  to  represent  Boolean  val¬ 
ues.  In  this  way,  the  similarity  of  two  BPOFs  were  determined,  and  d,-^[/i,m]  was  repre¬ 
sented  as  an  array  of +ls. 
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We  measured  the  similarity  of  BPOFs  across  the  entire  training  set  We  measured  the 
distances  between  coefficients  of  BPOFs  of  each  training  image  and  every  other  training 
image.  We  summed  all  the  distances  on  a  pixel-by-pixel  basis  and  used  this  quantity  as  a 
measure  of  the  clustering  of  values  for  a  particular  spatial  frequency.  We  used  the  criterion 
function^ 


N  N 


J[n,m]  = 

^  i=i;  =  l 


(3) 


to  measure  the  clustering  of  distances,  where  N  was  the  number  of  images  in  the  training 
set  A  more  compact  cluster  was  indicated  by  a  more  negative  value  at  a  particular  n  and 
m. 


By  removing  redundant  calculations  from  Eq.  (3),  the  clustering  was  ultimately  repre¬ 
sented  in  a  simpler  way.  For  example,  the  distance  between  a  filter  and  itself  yielded  an 
array  of  -Is.  In  addition,  the  XOR  function  is  communative,  and  the  distances  between 
two  filters  had  to  be  calculated  only  once.  We  found  that  the  clustering  of  distances  of  the 
training  set  could  be  represented  simply  by  a  summation  of  BPOF  coefficients  across  the 
training  set.  The  clustering  was  represented  by  the  function  J  [n,m]  as 


N 

J[n,m]  =  X  Hi[n,m] , 
i=  1 


(4) 


where  /  [n,m\  indicated  that  a  more  compact  cluster  was  indicated  by  larger  values.  This 
was  in  contrast  to  J{n,m],  where  a  more  compact  cluster  was  indicated  by  smaller  values. 

Our  main  objective  was  to  recognize  images  in  the  training  set  as  well  as  images  out¬ 
side  the  training  set  that  are  of  the  same  class.  Therefore,  we  selected  a  filter  that  retained 
features  (spatial  frequencies)  that  were  most  consistent  across  a  training  set.  Because  we 
have  a  measure  of  the  clustering  of  features  in  Eq.  (4),  we  retained  only  those  features  that 
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were  n30st  compactly  clustered  in  signal  space.  Using  this  approach,  we  expected  objects 
with  the  same  features  as  our  training  set  to  be  recognized  by  our  filter. 

We  used  factor  analysis^^  to  retain  spatial  frequencies  of  our  training  set  that  were 
more  compactly  clustered  than  others.  The  goal  of  factor  analysis  is  to  account  for  a  pro¬ 
cess  by  a  reduced  number  of  variables.  The  basis  of  factor  analysis  requires  an  assumption 
that  a  process  can  be  broken  down  into  additive  paits.^^  Because  the  process,  the  cross- 
correlation  of  a  traiiung  image  and  a  composite  filter  can  be  written  in  a  dis¬ 

cretized  form  as  the  summation 


n,  m 


(5) 


where  CQi  was  the  correlation  peak  value  for  the  tth  centered  training  image,  the  basic 
assumption  of  factor  analysis  was  satisfied.  The  spatial  frequencies  were  variables,  H{n.m] 
was  the  factor  at  n  and  m.  and  Sj[n,m]  was  the  factor  at  n  and  m,  in  the  tth  variate.  In  factor 
analysis,  usually  H[njn]  would  be  known,  S,[/i,ni]  would  be  estimated,  and  the  S,[/t7n]’s 
that  account  for  the  most  significant  variation  of  Cqi  would  be  retained. 

In  contrast  to  factor  analysis,  we  sought  to  retain  variables  n  and  m  that  did  not  signif¬ 
icantly  vary  C(^.  Therefore,  we  eliminated  the  spatial  frequencies  whose  coefficients  had 
large  distances  over  a  training  set  For  example,  small  values  of  J  [njn]  showed  that  some 
BPOF  coefficients  were  not  consistent  over  a  training  set  We  described  our  resulting  filter 
as 


1j/ y[n,m]  >p 

—\if  7[n,  m\  ^-p 
0  otherwise 

V 


(6) 


where  p  was  a  threshold,  and  a  =  p/N  was  a  factor  that  indicated  the  minimum  percentage 
of  features  across  the  training  set  that  had  the  same  value. 
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The  number  of  possible  different  filters  depended  on  the  number  of  images  in  the 
training  set  The  variable  p  could  take  values  from  0  ^  p  To  describe  different  filters 
with  each  value  of  p  and  to  avoid  choosing  a  value  for  G^/i^]  when  /  [njn]  was  0,  p 
should  be  odd  if  Af  is  odd,  ami  p  should  be  even  if  N  is  even.  The  number  of  different  pos¬ 
sible  filters  for  a  given  Af  was  described  as 


Af, 


'/itur 


N 

2 

N  1 
2  ■*‘2 


N  even 
N  odd 


(7) 


2.2  Performance  measures 

We  primarily  considered  the  SNR  and  the  peak-to-correlation  energy  (PCE)  as  mea¬ 
sures  of  performance.^  The  SNR  is  defined  as 


SNR 


|£{co)|^ 

var{cQ}  ’ 


(8) 


where  cq  is  the  correlation  peak  value  for  a  centered  in-class  image,  £{ }  is  the  expected 
value  and  var[ }  is  the  variance.  The  SNR  is  often  calculated  using  independent  noise  sam¬ 
ples  over  several  measurements  because  noise  will  make  the  correlation  result  change 
between  samples.  In  our  approach,  we  had  two  independent  sources  contributing  to  the 
change  in  the  correlation  result:  differences  between  training  images  for  unknown  reasons, 
and  additive  noise  due  to  the  sensor  or  correlator  system.  As  a  result,  the  SNR  will  be 
finite  without  additive  noise  present.  Consistent  with  Eq.  (5),  we  considered  S  [«,  m]  I 

*  *  ffl 

to  be  a  random  variable  with  an  expected  value  and  variance.  Therefore,  the  maximum 
value  of  Eq.  (8)  without  additive  noise  present  was  written  in  a  discretized  form  as 
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SNR  = 


_ _ _ _ 

5^  [Gg^[n,  m]]^var  {S  [n,  m]  }  +2]^  ^  G^[n^m\G^[n\m']cov  {S[n,m],S[n',m']  } 

Rl  ^  RiR  ^ 

(9) 


where  cov[ }  was  the  covariance  and  n,m<  n,  m’.  When  samples  of  5[/i^]  are  uncorre¬ 
lated,  Eq.  (9)  reduces  to 


SNR 


^  £  {5  [n,  ml  }  [a,  m] 

n^m _ 

^  [G^[n,m\]^var{S[n,m\  } 

R,  m 


(10) 


In  the  presence  of  additive  noise,  5[/i,m]  may  be  replaced  by  S[n,m]  +  P[n.m],  where 
P[njn]  is  the  power  spectral  density  of  the  noise.  Eq.  (10)  becomes 


SNR 


J)£{S[n,  m]  }C„[/i,m] 

n,  m _ _ 

X  [Ga  ['». '«]  1  {S[n,m]}  +  ^P[n,m]  [G^[n,m]f' 

n,m  n,m 


(11) 


The  PCE  can  be  used  as  a  measure  of  sharpness  of  the  correlation  peak  and  was 
expressed  as 
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PCE  =  p 


m]  C„[/i,m]| 
5^j5i[n»mJ|^|Co[n,m]|^ 


(12) 


for  a  particular 

3.0  Computer  simulations 

Infrared  (IR)  imagery  (8-14  pm)  of  ground  scenes  from  actual  sensors  were  used  to 
generate  and  evaluate  filters.  Images  were  digitized  with  128  x  128  pixels  with  8  bits/ 
pixel  Thresholding  was  performed  by  choosing  a  single  threshold  value  for  the  entire 
image.  In  the  imagery  we  examined,  the  background  and  object  were  easily  separated; 
however,  edges  were  not  well-defined.  As  variables  such  as  lighting,  noise,  and  atmo¬ 
spheric  effects  within  an  image  changed,  the  resulting  thresholded  images  would  have 
remained  similar  but  difTerent  We  used  objects  of  the  same  scale  and  rotation  to  examine 
the  effects  of  unknown  variations  of  the  inputs. 

We  used  15  uncoirelated  training  images  that  came  from  thresholded  sensor  imagery 
to  generate  different  statistical  filters,  one  for  each  possible  value  of  a.  Five  of  the  15 
images  are  shown  in  Fig.  1.  A  total  of  eight  filters  were  produced  with  values  of  a  =  0.07, 
0.20,  0.33,  0.47,  0.6,  0.73,  0.87,  and  1.00.  The  support  functions  of  the  filters,  the  region 
where  the  filter  was  transmitting,  are  shown  in  Fig.  2,  where  white  regions  indicate  the 
transmitting  areas.  The  support  function  when  a  =  0.07  is  not  shown  because  all  pixels 
were  transmitting.  We  evaluated  our  statistical  filters  using  images  firom  the  training  set 
and  additional  imagery.  We  used  both  types  of  imagery,  objects  that  were  not  in  the  train¬ 
ing  set  but  in  the  same  class  and  noisy  versions  of  the  same  images. 


3.1  Test  imagery  performance 

Using  the  training  set,  we  found  the  SNR,  PCE,  normalized  correlation  heights,  and 
transmitting  pixels  for  statistically-designed  filters  of  different  ot,  which  are  summarized 
in  Table  1.  The  correlation  heights  were  normalized  by  setting  the  maximum  correlation 
height  for  each  filter  equal  100.  Furthermore,  the  value  of  ^  in  Eq.  (12)  was  set  to  an  arbi¬ 
trary  constant.  In  general,  the  SNR  increased  as  a  increased,  while  the  maximum  SNR  of 
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258  occurred  at  a  =  0.73.  In  addition,  as  a  increased,  the  PCE  decreased.  The  results 
showed  that  the  correlation  responses  became  broader  but  more  consistent  as  a  increased. 

We  compared  the  results  of  our  statistically-designed  filters  to  that  of  an  fSDF  filter. 
An  fSDF  filter  is  an  iteratively  designed,  equal-correlation  peak  filter  that  takes  into 
account  spatial  light  modulator  (SLM)  modulation  characteristics.^  Therefore,  an  fSDF 
filter  was  designed  to  produce  equal  correlation  heights  for  the  training  set  using  a  BPOF. 
The  average  PCE  value  of  the  training  set  using  the  fSDF  filter  was  140.7,  which  was 
higher  than  the  values  obtained  with  the  statistically  designed  filters. 

For  correlation  filters  to  be  the  most  useful  with  sensor  imagery,  they  should  recognize 
objects  in  the  same  class  as,  but  not  in,  the  training  set  Therefore,  we  used  the  same  type 
of  imagery  from  which  the  training  set  was  gathered,  but  we  used  a  different  photograph 
to  generate  10  additional  test  objects  in  the  same  class  as  the  training  set  Furthermore,  we 
used  an  additional  nine  objects  that  visually  appeared  similar  to  the  training  set  but  were  in 
a  different  class  to  help  evaluate  the  discrimination  of  our  filters.  We  labeled  those  images 
that  were  not  in  the  same  class  as  the  training  set  as  images  26  -  29.  Four  of  these  images 
are  shown  in  Hg  3.  A  surmnary  of  the  cross-correlation  results  for  this  additional  imagery 
using  the  filters  in  the  previous  experiment  are  shown  in  Table  2.  The  data  showed  that  the 
SNRs  resulting  from  differences  in  the  correlation  heights  of  the  25  in-class  images  gener¬ 
ally  increased  as  a  increased  and  were  higher  than  the  fSDF  filter’s  response  for  most  val¬ 
ues  of  a.  The  average  normalized  correlation  heights,  using  in-class  imagery  not  in  the 
training  set  demonstrated  similar  behavior.  In  addition,  the  PCE  of  the  statistical  filters 
were  sometimes  comparable  but  always  lower  than  the  fSDF  filter.  We  used  the  ratio  of 
the  averages  of  the  normalized  correlation  heights  of  the  in-class  to  out-of-class  images  as 
a  measure  of  discrimination.  This  ratio  was  always  higher  with  the  fSDF  filter  but  compa¬ 
rable  to  the  statistical  filters  in  most  cases. 

3.2  Noise  performance 

We  repeated  the  previous  experiments  using  the  same  filters  but  with  noise  added  to  all 
test  images.  Ten  percent  of  the  pixels  in  each  test  image  were  randomly  selected  and 
changed  in  value  to  represent  noise.  The  previous  experiments  were  performed  with  noisy 
images,  and  the  results  were  summarized  in  Table  3.  We  compared  the  performance  of  the 
statistical  filters  to  a  ternary  version  of  an  fSDF  filter  specifically  designed  for  improved 
SNR  values  in  the  presence  of  noise  (referred  to  as  a  TSDFsj^  filter).^'^  For  the  statistical 
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filters,  the  SNRs  and  normalized  correlation  heights  generally  increased  as  a  increased. 
The  SNRs  of  the  statistical  filters  were  higher  than  the  TSDFsnr  filter  fcnr  most  values  of 
OL  The  normalized  correlation  heights  of  the  training  and  in-class  images  for  the  statistical 
filtos  were  always  higher  than  for  the  TSDFs]^  filters.  In  addition,  the  ratios  of  the  in- 
class  to  out-of-class  correlation  heights  were  more  similar  for  the  two  different  types  of 
filters  than  the  noise-fiee  case. 

33  Improved  SNR 

We  used  an  algorithm  that  has  been  successfully  used  to  design  optimal  POFs  for  addi¬ 
tive  noise^^  to  increase  the  SNR  of  our  statistical  filters.  We  sorted  the  values  of 
l£{5'[n,m]}l  after  forming  the  product  with  a  GJ^rijn];  different  thresholds  were  used  to 
determine  support  functions  for  a  particular  ol  All  possible  support  functions  were  found 
from  each  Gdn/n]  and  described  as 


/?„  =  {n,m;(£{5[n,m]}(|Ga[/»,m][  >r} 


(13) 


where  T  was  a  threshold.  We  found  the  maximum  SNR  and  corresponding  support  func¬ 
tion.  The  SNR  and  PCE  as  a  function  of  transmitting  pixels  according  to  Eq.  (13)  for  fil¬ 
ters  of  different  a  are  shown  in  Fig.  4.  In  all  cases,  the  results  show  that  the  highest  SNR 
values  occurred  at  relatively  low  PCE  values. 

In  some  applications  the  low  PCE  may  be  unacceptable.  Because  SNR  values  were 
often  satisfactory  for  a  variety  of  transmitting  pixels,  a  filter  may  be  chosen  that  is  not 
optimal  in  terms  of  SNR  and  PCE  but  may  yield  a  satisfactory  compromise. 

To  view  the  filters  in  terms  of  both  SNR  and  PCE  performance,  we  created  a  scatter 
diagram  in  Fig.  5  that  showed  the  SNR  as  a  function  of  PCE.  From  this  diagram,  the  filter 
that  had  the  highest  SNR  for  a  given  value  of  PC!E  can  be  chosen.  Filters  calculated  from 
Eq.  (6)  without  any  SNR  improvement  are  indicated  by  circles  around  their  data  points  in 
Fig.  5.  Note  that  these  filters  occurred  at  the  maximum  PCE  for  a  given  a.  Furthermore, 
some  filters  with  different  a’s  appeared  to  offer  similar  performance. 
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4.0  Experiment 


We  used  an  optical  correlator  that  is  described  in  more  detail  in  the  appendix. 

The  experimental  results  for  the  statistical  filter  for  a  =  0.07  both  when  noise  was,  and 
was  not  present  are  shown  in  Table  4.  The  main  difference  between  the  experimental  and 
simulated  data  was  that  the  values  of  the  correlation  heights  were  more  consistent  in  the 
experimental  data.  It  appeared  that  this  was  to  due  noise  associated  with  the  SLM.  When 
an  image  was  written  to  an  SLM,  a  small  percentage  of  the  pixels  (about  5%)  didn’t  dis¬ 
play  the  values  intended.  This  is  important  because  of  the  sensitivity  of  the  BPOF  to 
changes  in  an  input.  The  results  showed  that  the  correlation  heights  of  the  training  set  were 
less  than  expected  but  were  more  consistent 


5.0  Conclusions 

The  feature  that  contributed  most  to  the  improvement  in  SNR  of  correlation  filters 
when  compared  to  other  designs  was  the  statistical  design  methodology.  This  method  dif¬ 
fers  from  others  by  using  a  statistical  approach  to  identify  spatial  firequencies  useful  to 
increase  the  probability  of  recognition  of  in-class  objects,  especially  those  not  in  the  train¬ 
ing  set  The  performance  of  the  statistically-designed  BPOFs  depended  on  a  parameter  a. 
The  SNR  and  normalized  correlation  heights  of  objects  generally  increased  as  a 
increased.  In  most  cases,  the  SNR  and  normalized  correlation  heights  of  the  statistically- 
designed  filters  was  higher  than  both  the  fSDF  and  TSDFsnr  filters.  Similar  results  were 
found  when  the  statistical  filters  were  tested  with  noisy  test  images 

Another  feature  of  our  approach  that  contributed  to  the  improvement  of  SNR  filters 
was  the  retention  of  spatial  frequencies  based  on  their  magnimde.  By  combining  this 
method  with  our  statistical  approach,  we  could  maximize  the  SNR  for  a  given  value  of 
PCE. 

The  statistically  designed  filters  were  more  easily  calculated  than  both  fSDF  and 
TSDFsnr  filters.  The  statistical  filters  required  calculating  N  FFTs.  In  contrast,  the  fSDF 
approach  required  a  cross-correlation  between  every  training  image  and  filter  for  every 
iteration.  This  approach  requires  NI FFT  calculations,  where  I  is  the  number  of  iterations 
and  generally  has  been  set  to  10.^  The  TSDFsnr  filter  required  2NI  FFT  calculations. 
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Therefore,  the  tiine  and  number  of  operations  required  to  calculate  our  statistical  filters 
were  about  an  order  of  magnitude  less  than  that  of  the  fSDF  and  TSDFsnr  filters. 

The  sensitivity  of  a  Bf^DF  to  changes  in  an  object’s  appearance  is  a  significant  factcn’ 
when  attempting  to  recognize  nondeterministic  objects  of  the  same  class.  Eliminating 
properly  selected  spatial  frequencies  can  reduce  this  sensitivity.  However,  this  will  affect 
discrimination.  In  our  experiments,  the  ratios  of  correlation  heights  for  in-class  to  out-of¬ 
class  images  were  generally  lower  for  the  statistically-designed  filters  when  compared  to 
other  approaches.  In  addition,  correlation  responses  were  generally  less  sharp  and  the  light 
efficiency  was  lower  for  the  statistical  filters  when  compared  to  other  approaches. 

Correlation  is  a  basic  operation  that  is  required  by  neariy  all  machine  vision  and  pat¬ 
tern  recognition  systems.  As  larger  and  faster  input  sensors  are  used,  the  speed  of  elec¬ 
tronic  systems  is  stretched  to  beyond  its  limit  By  utilizing  the  massive  parallelism  and 
high  bandwidth  offered  by  the  optics  technology,  much  more  data  can  be  processed  at 
higher  rates.  The  optical  correlator  presents  a  nearly  mature  technology  that  needs  work  in 
the  areas  of  device  development  and  system  packaging  for  uses  in  either  military  or  civil¬ 
ian  applications.  A  growing  number  of  manufacturing  industries  are  looking  towards 
machine  vision  to  improve  the  performance  and  capabilities  of  automated  assembly  lines 
and  inspection  techniques. 

It  is  recommended  that  for  objects  to  be  recognized  from  sensor  imagery  using  an  opti¬ 
cal  correlator,  emphasis  should  be  placed  on  reducing  the  trade-offs  between  robusmess, 
and  discrimination,  and  light  efficiency. 
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Appendix 


This  appendix  describe  some  important  characteristics  of  the  correlator  such  as  the 
experimental  apparatus,  the  alignment,  and  the  controlling  and  displaying  of  images.  The 
correlator  uses  binary  SLMs,  so  we  used  binary  phase-only  filters.  Images  and  filters  were 
generated  on  a  computer  and  put  in  the  proper  format  for  display  on  the  SLMs.  The  system 
is  a  research  platform  so  it  may  need  realignment  from  time  to  time,  or  some  procedures 
may  seem  cumbersome. 

We  used  two  set  -ups  that  were  pretty  much  the  same  in  terms  of  performance.  We  first 
set  -up  a  correlator  using  a  He-Ne  laser  because  of  the  known  good  beam  quality  of  this 
device.  Because  the  SLMs  used  were  highly  attenuating,  the  correlation  result  was  often 
dim.  More  powerful  He-Ne  lasers  (>15mW)  are  typically  quite  long,  so  we  used  a  coir- 
pact  higher  power  frequency-doubled  YAG  0=532nm)  laser  in  another  set-up. 

The  laser  in  the  first  set-up  was  a  Mellis  Griot  linearly  polarized  He-Ne  operating  at  a 
wavelength  of  633nm;  we  measured  13mW  at  the  output  of  the  laser.  The  spatial  filter 
assembly  was  made  by  Newport  and  consisted  of:  a  40x  microscope  objective,  10mm  pin¬ 
hole,  and  250mm  focal  length  collimating  lens.  The  correlation  result  can  often  be  quite 
dim,  especially  when  using  the  fran^grabber  to  capture  a  result;  therefore,  sometimes  a 
20x  objective  was  used  or  we  removed  the  pinhole.  The  mirror  is  oriented  at  about  45 
degrees  and  is  used  simply  to  bend  the  beam  around  the  optical  bench.  Unfortunately,  the 
mirror  may  produce  an  elliptically  polarized  reflected  beam  from  the  linearly  polarized 
incident  beam. 

The  laser  in  the  second  set-up  was  a  Adlas  200  diode-pumped,  frequency  doubled 
YAG  laser  operating  at  a  wavelength  of  532nm.  The  spatial  filter  assembly  was  made  by 
Newport  and  consisted  of:  a  20x  microscope  objective,  10mm  pinhole,  and  250mm  focal 
length  collimating  lens.  We  found  that  the  pinhole  must  be  used  for  meaningful  results. 

The  elliptically  polarized  beam  can  be  compensated  for  with  a  quarter- wave  plate  (the 
light  entering  the  SLM  should  be  linearly  polarized).  To  determine  if  a  quarter-wave  plate 
is  needed,  a  polarizer  should  be  placed  temporarily  in  the  beam  after  being  reflected  from 
the  mirror.  The  polarizer  should  be  rotated  so  that  the  beam  intensity  is  a  minimum.  If  this 
minimum  is  different  than  the  minimum  obtained  with  the  polarizer  placed  before  the  mir¬ 
ror,  then  the  beam  may  need  to  be  compensated. 
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To  adjust  the  quarter-wave  plate,  place  the  polarizer  after  the  mirror  and  adjust  it  so 
that  it  produces  a  minimum.  Then  place  the  quarter-wave  plate  in  the  path  of  the  beam 
before  the  mirror.  Rotate  the  quarter-wave  plate  so  that  a  minimum  is  obtained  at  the  out- 
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put  of  the  polarizer.  Now  the  light  entering  the  input  SLM  is  linearly  polarized.  The  quar¬ 
ter-wave  plate  produces  an  elliptically  polarized  beam  that  is  cancelled  by  the  mirror.  We 
found  that  for  the  second  set-up,  the  quarter-wave  plate  was  mandatory. 

The  input  SLM  is  imaged  onto  the  filter  SLM  by  lenses  1  and  2.  The  calculation  of  the 
effective  focal  length  required  and  the  alignment  procedure  will  be  discussed  in  the  fol¬ 
lowing  paragraphs.  The  effective  focal  length  required  was  1168mm  for  set-up  1  and 
1340mm  for  set-up  2.  It's  difficult  to  find  a  lenses  of  exactly  these  focal  lengths,  so  we 
used  two  lenses.  In  addition,  the  SLM  should  be  placed  a  distance  equal  to  a  focal  length 
before  the  Fourier  transform  lens  for  best  imaging.  Our  experience  shows  that  this  require¬ 
ment  can  be  relaxed;  besides,  such  a  system  takes  up  more  space  and  needs  more  mirrors. 
We  both  set-ups  we  used  an  achromatic  lens  of  f  =  1000mm  for  lens  1  and  a  plano-con¬ 
cave  lens  of  f  =  -  250mm  for  lens  2. 

The  input  SLM  is  operated  in  a  maximum  contrast  mode.  That  means  that  the  back¬ 
ground  of  an  image  displayed  on  the  SLM  is  adjusted  (with  the  polarizer  on  the  SLM)  so 
that  a  minimum  of  light  comes  through. 

The  filter  SLM  is  operated  in  a  phase-only  mode.  To  adjust  the  filter  SLM  to  operate  in 
the  phase-only  mode,  the  light  fiom  the  input  SLM  must  be  incident  on  the  filter  SLM. 
When  rotating  the  polarizer  on  the  filter  SLM,  there  is  a  region  where  the  pattern  on  the 
filter  SLM  quickly  "flips";  regions  on  the  SLM  that  were  transmitting  become  opaque  and 
vice  versa.  The  point  in  between  these  different  contrast  modes  where  there  is  no  contrast 
is  the  phase-only  point  of  operation. 

The  correlator  is  a  relatively  simple  device;  however,  its  aligimient  is  critical.  The 
most  critical  factor  in  its  alignment  is  the  imaging  of  the  input  SLM  onto  the  filter  SLM. 
The  rule  that  governs  this  part  of  the  alignment  is  that  the  highest  spatial  frequency  of  the 
input  SLM  should  be  imaged  onto  the  edge  of  the  filter  SLM.  Therefore,  the  focal  length 
of  the  Fourier  Transform  lens  should  be^ 


f=NdidxfK 


(14) 
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where  N  is  the  number  of  pixels  along  a  side  of  the  filter  SLM  (128),  d2  and  di  are  the  cen- 
ter-to-center  spacing  of  die  pixels  of  the  filter  and  input  SLMs  respectively  (76mm),  and  X 
is  the  wavelength  of  light  being  used.  The  highest  spatial  frequency  grid  should  be  placed 
on  the  input  SLM.  The  grid  will  be  rows  of  dark,  light,  dark,  light,  etc.  The  input  SLM  and 
the  lenses  should  be  adjusted  so  that  the  first  diffraction  order  due  to  the  pattern  on  the 
input  SLM  is  on  the  edge  of  the  filter  SLM. 


The  two  Semetex  SLMs  the  correlator  uses  are  made  by  the  same  company  but  share 
little  between  them.  The  devices  use  different  software,  are  driven  by  different  cards,  and 
accept  different  types  of  file  formats  for  images  that  may  be  displayed  on  them.  The  SLM 
in  the  filter  plan  is  an  “older”  device  of  128  x  128  pixels  and  the  SLM  in  the  input  plane  is 
a  “newer”  device  256  x  256  pixels.  The  newer  devices  do  not  display  an  intricate  pattern 
as  reliably  as  the  old  devices  do.  This  is  important  for  a  filter  SLM  because  the  pattern  is 
usually  intricate.  Therefore,  we  use  the  old  device  in  the  filter  plane.  In  addition,  the  image 
displayed  on  the  input  SLM  should  only  occupy  128  x  128  pixels  or  distortion  and  an 
overlapping  of  the  correlation  responses  will  result  Finally,  it's  not  clear  what  must  be 
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done  to  drive  both  devices  from  the  same  computer.  There  may  be  a  conflict  with  the 
driver  boards  but  this  problem  should  not  prove  too  difficult  However,  we  currently  use  a 
separate  computer  to  drive  each  SLM. 

There  exists  some  compatibility  problems  with  generating  an  image  to  be  displayed  on 
an  SLM  and  then  actually  having  it  displayed.  This  problem  arises  because  the  SLMs  must 
be  driven  with  IBM-compatible  computers,  and  image  manipulation  and  generation  has 
often  been  performed  on  the  Macintosh.  The  IBM  computers  have  generally  been  harder 
to  work  with  than  the  Mcintosh  in  terms  of  the  graphics  needed  to  generate  images  and  fil¬ 
ters.  Maybe  in  the  future,  all  development  of  imagery  and  filters  could  be  done  on  the  IBM 
machines,  but  that  is  not  the  case  now. 

Although  some  compatibility  problems  exist,  they  are  not  prohibitive.  Software  has 
been  developed  to  help  ease  the  transfer  of  Macintosh  images  to  the  proper  format  for  the 
IBM.  Let's  start  with  the  IBM  computers  and  examine  how  we  can  display  inuiges  on  the 
SLMs.  Well  discuss  the  specific  programs  to  drive  the  SLMs  later,  an  important  factor  is 
that  an  image  to  be  displayed  on  an  SLM  must  be  in  the  proper  format,  the  .bnd  format  for 
the  old  device  (filter  plane)  or  the  .sen  format  for  the  new  device  (input  plane).  Because 
the  Macintosh  has  been  so  useful  in  generating  images,  more  effort  has  been  put  input 
converting  Macintosh  images. 
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TABLE  1.  PerformaiKe  of  statistically  designed  fiUters  for  images  in  training  set 


Statistical 

aMim 

0^.2 

oe»033 

0-0.47 

<p^j6 

0-0.73 

O-037 

a-1.00 

Tiansmioing  pixels 

16384 

11276 

1031 

4220 

2931 

1858 

1324 

918 

SNR  of  training  set 

6S.6 

79.8 

992 

144 

214 

258 

222 

202 

Avg.  PCE  of  training  set 

131 

136 

131 

113 

94.6 

92.5 

dsn 

67.4 

Avg.  nonnalized  conela- 
tion  height  of  training  set 

75.1 

75.7 

81.4 

84.0 

84.8 

873 

88.9 

TABLE  2.  Performance  of  filters  for  images  of  in-dass  and  out-of  class  sets 


Statistical 

fSDF 

filter 

a-0.07 

aM)2 

O-033 

O-0.47 

O-0.6 

O-0.73 

0-0.87 

o— 1.00 

SNR  of  in^lass  set 

27.0 

30.1 

40.9 

52.8 

86.1 

86.7 

863 

45.7 

Avg.  PCE  of  in-class  set 

86.8 

90.7 

883 

76.8 

64.8 

62.9 

56.7 

46.4 

90.8 

Avg.  nonnalized  correla¬ 
tion  height  of  in-class 
images  (not  in  training 
set) 

48.0 

493 

59.4 

65.6 

72.2 

76.5 

76.6 

75.7 

58.8 

Ratio  of  average  in-class 
to  out-of-class  correlation 
heights 

3.88 

3.97 

4.67 

4.82 

4.49 

4.11 

3.76 

339 

6.00 

TABLE  3.  Normalized  correlation  heights  for  filters  using  images  with  10%  noise 


Statistical  niter 

TSDFsnr 

filter 

O-0.07 

0-03 

0=033 

0-0.47 

O-0.6 

0=0.73 

O-0.87 

o-l.OO 

SNR  of  training  set 

53.4 

68.5 

80.4 

no 

195 

209 

185 

200 

70.1 

Avg.  nonnalized  cone- 
iadon  height  of  train¬ 
ing  set 

623 

53.0 

516 

553 

59.2 

61.0 

60.4 

619 

33.9 

SNR  of  in-class  set 

21.9 

27.0 

37.8 

48.6 

52.9 

713 

77.0 

717 

63.9 

Avg.  nonnalized  corre¬ 
lation  height  of  in-dass 
images  (not  in  training 
set) 

40.0 

38.7 

47.4 

473 

47.4 

53.3 

53.8 

53.9 

30.9 

Ratio  of  average  in- 
class  to  out-of-class 
ctKielation  heights 

120 

2.65 

3.69 

197 

3.11 

3.22 

3.44 

3.21 

3.68 

TABLE  4.  Experimental  results  for  a  =  0.07 


Statistical  filter 

10% 

noise 

SNR  of  training  set 

102 

65.7 

Avg.  nonnalized  axiela- 
tion  height  of  training  set 

90.8 

73.4 

SNR  of  in-class  set 

32.7 

30.5 

Avg.  nonnalized  conela- 
dan  height  of  training  set 

(not  in  training  set) 

1.77 

1.43 

Image  1  Image  2  Image  3  Image  4  Image  5 


FIGURE  1.  Examples  of  images  used  in  training  set  obtained  from  sensor  data. 
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Image  26  Image  27  Image  28  Image  29 

FIGURE  3.  Examples  of  out-of-class  images  from  sensor  data. 
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FIGURE  4.  Graphs  for  different  a  of  (a),  (b)  SNR  as  a  function  of  transmitting  pixels, 
(c),  (d)  PCE  as  a  function  of  transmitting  pixels. 
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FIGURE  5.  Scatter  diagram  for  SNR  and  PCE  for  different  a  using  the  training  set 
Note  that  circled  data  points  indicate  filters  calculated  directly  from  Eq.  (6). 
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FIGURE  6.  Schematic  diagram  of  optical  correlator 


30 


MISSION 

OF 

ROME  LABORATORY 


Rome  Laboratory  plans  and  executes  an  interdisciplinary 
program  in  research,  development,  test,  and  technology 
transition  in  support  of  Air  Force  Command,  Control, 
Communications  and  Intelligence  (C3I)  activities  for  all 
Air  ^ Force  platforms.  It  also  executes  selected 
acquisition  programs  in  several  areas  of  expertise. 
Technical  and  engineering  support  within  areas  of 
competence  is  provided  to  ESC  Program  Offices  (POs)  and 
other  ESC  elements  to  perform  effective  acquisition  of 
C3I  systems.  In  addition,  Rome  Laboratory's  technology 
supports  other  AFMC  Product  Divisions,  the  Air  Force  user 
community,  and  other  DOD  and  non-DOD  agencies.  Rome 
Laboratory  maintains  technical  competence  and  research 
programs  in  areas  including,  but  not  limited  to, 
communications,  command  and  control,  battle  management, 
intelligence  information  processing,  computational 
sciences  and  software  producibility,  wide  area 
su^eillance/sensors,  signal  processing,  solid  state 
sciences,  photonics,  electromagnetic  technology, 

and  electronic 
rsliability/ittaintain^bility  and  testability. 


